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ABSTRACT 

A preliminary report on a long-term empirical 
investigation of advanced language acquisition in a study-abroad 
environment is presented. The project is based on an American Council 
of Teachers of Russian (ACTR) longitudinal study of the gains in 
Russian language competence demonstrated by American students in 
semester-long language programs in Moscow and Leningrad. Six-year 
data include student information on age, gender, citizenship, country 
of birth, place and levels of formal education, and information on 
all prior training in the Russian language. Analysis of test results 
in oral, listening, and reading proficiency shows that the immersion 
experience produces significant gains in language proficiency 
achieved by a variety of American college-level learners of Russian. 
One of the conclusions is that at any given initial level, the better 
prepared the student is on reading/grammar the more the student tends 
to gain. Also, the Modern Foreign Language Aptitude Test (MLAT) is 
shown to be a valid predictor of success in developing reading and 
listening skills in Russian, although it does not serve to predict 
oral proficiency gain as measured by the standard Oral Proficiency 
Interview (OPI) . Another finding is that the gains are significantly 
higher for males than for females, a phenomenon that may be 
influenced by communication interactions in-country. Contains 13 
references. (LB) 
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Oa Introduction 

The present study constitutes a preliminary report on a long- 
term empirical investigation of advanced language acquisition in a 
study-abroad environment. The report consists of four parts: Section 
One sketches the broad context of cognitive research which has 
motivated and guided this study. Section Two describes the project 
itself, its scope, methodology, goals, and the specific variables 
examined. Section Three contains Sie analysis of the data, and Section 
Four offers a preliminary discussion of these results. 

This paper was presented in Russian at the VII International Congress of MAPRIAL, 
Moscow, August, 1990. 



Richard D. Brecht, University of Maryland 
Dan E. Davidson, ACTR and Bryn Mawr College 
Ralph B, Ginsberg, University of Pennsylvania 
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The project has been funded by the United States Department 
of Education with the collaboration and support of the American 
Council of Teachers of Russian (ACTR) and the National Foreign 
Language Center (NFLC). It is based on an ACTR longitudinal study 
of the gains in Russian language competence demonstrated by 
American students in semester-long language programs in Moscow 
and Leningrad J 



1.0 Guiding Principles 

The long-range, empirical study of the **value added** by 
immersion experience in a study-abroad environment is a centrsJ 
element in the second language acquisition research agenda. Empirical 
in its approach and anchored within the research agenda of 
contemporary cognitive science, the study has implications for foreign 
language and international studies policy. 

1.1 Policy Ramifications of Empirical Research 

Whether charged with starting a new program or evaluating 
and revising an existing one, whether making sweeping curricular 
changes or deciding on a specific course offering for a particular 
semester, program directors and foreign language administritors at all 
levels frequently set and implement policy. 

Wherever possible, foreign language training policy should be 
formulated and guided by empirical research on second language 
acquisition.^ The following questions exemplify the kinds of issues 
policymakers may address to tihe research community: 

• What adjustments should be made between 
domestic and study-abroad learning, 
between beginning and advanced level 
learning? 

• What is the best and most cost-effective 
study-abroad program? How can a program 
be evaluated? 



102 



On Evaluating Language Proficiency Gain in Study Abroad EnvironmeDts 



Richard D. Brccht/Dan E. Davidson/Ralph B. Ginsberg 



• Should students be advised of probabilities 
of their success in a study-abroad program? 

• Should programs abroad be adjusted to 
accommodate different types of students: 
majors vs. non-majors, literatureAinguistics 
vs. areas studies majors, humanities vs. 
science majors? If so, at what level of 
acquisition should **language for special 
purposes" be undertaken? 

• Should the study-abroad experience be an 
obligatory part of an undergraduate FL 
program, and where in the program does it 
most appropriately fit? 

• Is study-abroad more or less cost-effective 
than domestic instruction? 

• Is the process of foreign language 
acquisition in an in-country immersion 
environment different from domestic 
classroom instruction, and hov/ is it 
different?3 

• What level of proficiency is expected of 
students finishing a study-abroad ^program? 
Begirming a program? 

• Are the structured, semi-structured, and 
unstructured environments organized, 
supported, and interrelated? 

• Do testing and evaluation adequately reflect 
the skills acquired abroad? 



These questions represent only a sample of the kinds of 
information required by program advisors and policymakers. 
Although researchers cannot provide definitive answers to many of 
the above questions, the present study is pertinent to several of them. 



X.2 The Research Agenda for 

Second Language Acquisition (SLA) 

While policy questions affect the need for research on study 
abroad, our scientific concern is to provide empirical evidence for the 
efficacy of this mode of language acquisition. Our general approach is 



103 



On Evaluating Language Ptoficiency Gain in Study Abroad Environments 
Richard D. Brecht/Dan £. Davidson/Ralph B. Ginsberg 



grounded in the relatively new and developing fields of SLA and 
"cognitive science/' an amalgam of related concems in psychology, 
computer science, linguistics, philosophy, and neuroscience. The 
research agenda for SLA is still evolving, just as it is for cognitive 
science as a whole,^ and is concerned with the nature of knowledge of 
L2 (declarative and procediural) and the nature of learning. Given that 
the dependent variable of the present study is a measurement of gain 
derived from pre-and post-program psychometric instruments, the 
research presented here is best viewed from the perspective of 
learning as a process of input, internal processing, and output.^ A 
general overview of SLA is presented in Exhibit One. 

The variables in the present study are concerned with input 
(beginning level of L2 knowledge and learner differences) and output 
(post-program assessment of L2 knowledge). Exhibit One makes 
clear, though, that there is much more to be measured in input, 
particularly the contexts of learning and the language used and heard, 
and much to be investigated with regard to the psychological and 
cognitive factors involved in the learning process itself. 



2. The ACTR/NFLC Project 

The goal of the ACTR/NFLC project is the determination of 
factors which correlate with "gain" in Russian language ability as a 
result of long-term study in the Soviet Union.^ The data bank serving 
as the base of this study was established by ACTR, an organization 
which, as a part of its overall professional mission, executes the 
largest academic exchange program between the Soviet Union and the 
United States (see Exhibit Two). For 15 years, records of university 
students studying Russian for one semester in, at present, eight Soviet 
institutes in Moscow and Leningrad have been systematically 
converted to machine readable form using a modified form of the 
"Paradox" data base system. Records include the material on the 
student application forms, data gathered in the regular common 
briefing and debriefing sessions, transcripts from Soviet institutes, and 
reports from ACTR academic program officers in the Soviet Union. 

From the beginning, the data collected by ACTR has not been 
limited to that needed for executing the exchanges; rather, academic 

5 
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policy ramifications were always a consideration^ The present project 
was designed, first, to establish the b^^^is for detennining the success 
of a program with regard to quality and cost-effectiveness and to 
evaluate existing programs; second, to improve program design, 
implementation, and cost-effectiveness; third, to determine the best 
predictors of success in a language learning career, particularly at the 
more advanced levels, with the ultimate goal of selection and 
placement of students in specific programs. A further goal was added 
to the present USED-funded project, that is to estabUsh and make 
available to different (particularly less-commonly taught) languages a 
model for the collection of empirical data on SLA. 
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Exhibit One 
An Overview of SLA 



Situation. 



, Structured 
-Unstructured 



1. Input: 



External 
(Observable^^ 




Data Ini 




Cognitive^ 



Internal 
(Psychological) 



'Learner differenfigs 



Language, 
Knowledge 



. L1 

u 



/ 



n. Interna' 
Proceaalnjis 



Cognltiv 



' Background knowledge 



Personal 



General 
Age 
Gender 
Aptitude 
Cognitive style 
Motivation 



social (stmlagies to mana,e interaction in 12) P-rsonaiity 




Production/Reception 



Communication 



IIL Outpu^^^ 



Natural (Observable Irjteractlons. 
interpretations, texts) 

• Controlled (Testing instruments) 



ERIC 



^ SESTCOPYAVAIIABIE 



7 



On Evaluating Language Proficiency Gain in Study Abroad Environments 
Richard D. Brecht/Dan E Davidson/Raiph B. Ginsberg 



2.1 Classification of Variables 

The starting point for this study is the output, specifically the 
measurement of L2 **gain/* by American students in a number of 
ACTR-affiliated institutions in Moscow and Leningrad. Since 1976, 
ACTR has maintained records pertaining to the general academic, 
biogr^hical, and in-country language performance of more than two 
thousand American students and graduate students who have 
completed long-term language training programs in the USSR under 
ACTR auspices. During most of the recording period, participants 
have typically been at die B.A. or inmiediate post-B.A. level, 22.2 
years of age, with undergraduate major or equivalent preparation in 
Russian language and area studies, with or without other academic 
specializations. Due to previous limitations imposed by the Soviet 
government on the number of Americans permitted to study in the 
USSR, competition in the US for places in the ACTR programs was 
keen with as many as 4-6 qualified Russian majors or graduate 
students in the field applying for each position abroad. The resulting 
escalation of de facto qualifying standards worked to the advantage of 
students from institutions with intensive summer training and 
considerable advanced-level formal course work (competency-based 
language courses beyond the third-year college level), effectively 
limiting to a significant degree the number of participants from 
smaller Russian departments throughout the US.^ 

A steady increase in the number of positions for study in the 
USSR since 1985 (from 110 persons in 1984-85 to 520 persons in 
1989-1990) has been accompanied by a doubling in the number of 
sending institutions in the US, reflecting a greater diversity of 
institutional types and geography than was the case in earlier years. At 
the present time a total of 195 colleges and universities have placed 
students or faculty in the ACTR programs. Exhibit Two lists 
American Colleges and Universities arranged in order of the number 
of students placed in ACTTR long-term (primarily semester) programs 
between 1976 and 1989. In the case of participants who have either 
graduated or received their training at more than one institution, the 
current or most recent institutional affiliation is cited. The listing 
reveals a range of geographically diverse public and private 
institutions including small colleges and large comprehensive 
universities. A diversity of types of Russian language training 
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programs are represented among the top 65 institutions, no one of 
which accounts for more than six percent of the total ACTR data base. 

The six-year sample selected for analysis in the presc . study 
includes background data on age, gender, citizenship, country ot birth, 
place and levels of formal education (including highest degree taken), 
major field(s), and information on all prior training in the Russian 
language, including number of contact hours, prior experience abroad, 
experience in intensive stateside courses, secondary school programs, 
language laboratory, and fcaowledge of other Slavic and non-Slavic 
languages. Also included in the data base for each student record is 
the academic year and type of program completed (sunmier, semester, 
academic year) and host institution in the USSR. (See Exhibit Three: 
ACTR Student Record System Semester Program Data Directory, 
1984-89). 

Pre- and post-program standardized tests include (since 1984) 
oral proficiency interviews (OPI) administered by ACTFL or 
government-certified specialists trained in the elicitation of Russian 
speech samples rateable in terms of a common five-point metric of 
functional competency in the language (see Exhibit Four). Pre- and 
post-program OPI testing is normally recorded on audio cassettes, 
and, in certain cases, has been subjected to reconfirmation by 
additional certified testers. Comparable proficiency-based 
standardized testing for reading and listening in Russian was 
developed by the Educational Testing Service (ETS) in cooperation 
. with the Russian language teaching field and introduced in tiie mid- 
1980s. Available in two forms and administered under secure and 
controlled conditions, these standardized tests are machine-graded and 
reported regularly by ETS directly to ACTR where they are added to 
the student records data base both as weighted raw scores and as 
proficiency-rating conversions, pre- and post-program, for all 
participants. 
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Exhibit Two 



College and University Participation in ACTR Programs 

Top 65 Colleges Arranged by Number of ACTR Program Participants 
Semester Programs 1976-89 





Number of 






Program 




Academic Affiliation 


Participants 


Academic AffiUatlon 


Bryn Mawr CoUcgc 


68 


New Hampshire, University of 


Harvird University 


60 


Reed CoUege 


Pennsylvania, Univcrjity of 


50 


Triruty CoUege — Connecticut 


Columbia University 


41 


California. University of — Davis 


Georgetown University 


37 


George Mason University 


Yale University 


36 


California, University of — Los 


Grinnell College 


31 


Angeles 


Iowa, Univenity of 


30 


Minnesota, University of 


Northwestern Univeisity 


30 


PriiKipia CoUege 


Wesleyan University 


30 


Vermont. University of 


Michigan, University of 


29 


Mount Holyoke CoUege 


Amherst CoUege 


26 


Triiury University 


Brown Univctsity 


25 


James Madison University 


Maryland, University of — College 




New York University 


Park 


23 


DattmouA CoUege 


Indiana University 


21 


WeUesIey CoUege 


Oregon, University of 


21 


Maryland, University of — Baltimore 


Princeton University 


20 


County 


George Washington Univeisity 


18 


Massachusetts, University of — 


California, University of Berkeley 


18 


Amherst 


Williams CoUege 


17 


Vanderbilt University 


Texas, University of — Austin 


17 


American University 


ComeU University 


15 


New Mexico, University of 


niinois, Uni\'ersiiy of — Urbana 


14 


North Carolina, University of — Chapel 


Haverford CoUege 


13 


HiO 


Middlebury College 


13 


North Iowa, University of 


Wisconsin, University of 


13 


Tufts University 


Virginia, University of 


13 


Monterey Institute of International 


Bowdoin CoUege 


13 


Sudies 


Chicago, Univeisity of 


12 


Oklahoma State Univetsif" 


California, University of — Sanu Cruz 


12 


Pomona CoUege 


Stanford University 


12 


Illinois, University of — Chicago 


Brigham Young Universi^ 


11 


Arizona, University of 


Carleton CoUege 


10 


Rutgers University — Rutgers CoUege 


Swazthraocc CoUege 


9 


Barnard CoUefe 


Washington, University of 


9 


100 Other Colleges and Univtrsities 



Number of 
Program 
Participants 

8 
8 
8 
8 

8 



Total 



6 

6 

6 
6 
6 
5 

5 
5 
5 

5 
5 
5 
5 
5 
5 
5 

198 
1195 



lERlC 



10 

BEST COPY AVAIUBIE 



109 



On Evaluating Language Proficiency Gain in Study Abroad Environmtnts 
Richard Brecht/Dan R Davidson/Ralph B. Ginsberg 



Exhibit Three 



ACTR Student Record System 
Semester Program Data Only, 1984-89 

Data Dictionary — Listing of Student Record Fields Recoded and Transferred 



Field Nftme 



Rimge 



Descriptive Note 



Program Year 

Program Type 

Highest Degree 

Major of Highest Degree 

Study Institute 

Date of Birth 

Age 

Sex 

Country of Birth 
Years High School Russian 
Years College Russian 
Total Oass HoursAVcck 
Total Lab Hours/Week 
Number of Slavic Languages 
Number of Non-Slavic Luig. 
Number of Prev. USSR Immr 



1984-89 

M 
1-6 
1-6 

17-38 
1-2 

1, 2. 3. 5 

0-9 
0-152 

0-1 
0-6 

0-3 



Total of 466 students ranging over 6 years 
Semester program smdents in Fall or Spring 
Sec Code Listing Report 
See Code Ustmg Report 1 = Russian... 
See Code Listing R4>ort 1 = Pushkin... 



Assessed as age at application 

1= Female. 2 = Male 

1 « USA, 2 = Canada, 3 = Europe, 5 = 



Other 



Sum of hours per semester 

Sum of hours per semester 

Number of other Slavic languages smdied 

Number of non-Slavic languages smdied 

Number of immersion studies in Soviet Union 



MLATin 

MLvriv 

MLATV 

MLAT Short Form Raw 



QujJify Grammar 
Oualiiv Reading 



Continuous 
Continuous 
Continuous 
Continuous 



Modem Language Aptimde Test Section 
Modem Language Aptimde Test Section 
Modem Language ^timde Test Section 
MLATm+MIATiy-fMLATV 



Continuous 
Continuous 



Prc-ETS Listening 
Pte-ETSListPirofic 
Post-ETS Listening 
Post-ETSUstProfic 
ETS Listening Change 
Prc-ETS Reading 
Pie-ETSReadProfic 
Post-ETS Reading 
Post-ETS Read Ptofic 
ETS Readine Change 



Continuous 

to 40, 03-07 

Continuous 

01-07 

Continuous 

Continuous 

01-07 

Continuous 

01-07 

Continuous 



Number right/Base 
Number right/Base 



05=2,06 = 
05=2,06 = 



?rc-Oral Proficiency 
Post-Ora) Proficiency 
Intellectual Motivation 
Natural Ability to Leam 
Willingness — Use Russian 
Cultural Adapubility 
Willingness—Cultural 0pp. 
Ability to work in group 
Leadership Potential 



01-07 
01-07 



Current ETS tests begun in 1986 
01 = 0. 02 = 0+, 03 = 1, 04 = 1+, 

01=0.02 = 0+, 03 = 1,04 = 1+, 
Difference in Post-Pre plus 100 



01 = 0. 02 = 0+, 03 = 1 . 04 = 1+. 05 = 2, 06 = 

01 = 0. 02 = 0+. 03 = 1 , 04 = 1+, 05 = 2, 06 = 
Difference in Post-Pre glus 100 



2+, 07 = 3 
2+, 07 = 3 

2+, 07 = 3 
2+, 07 = 3 



0-5 
C-5 
0-5 
0-5 
0-5 
0-5 
0-5 



01 = 0, 02 = 0+, 03 = 1. 04 = 1+, 05 = 2, 06 = 2+, 07 = 3 
01 =0. = 0+. 03 = 1. 04 = 1+, 05 = 2. 06 - 2+. 07 = 3 



Director's Evaluation 0 = Low, 5 = High 
Director's Evaluation 0 = Low, 5 = High 
Director's Evaluation 0 Low, 5 = High 
Director's Evaluation 0 = Low, 5 = High 
Diicctor's Evaluation 0 = Low, 5 = Hi|^ 
Director's Evaluation 0 = Low, 5 = High 
Director's Evaluation 0 = Low. 5 = High 



Records filtered by the three following rules: 

Student age at application tnust be less than 40 years. 
Student must not be bom m Slavic speaking country. 
Student must not speak more than one other Slavic language. 
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Exhibit Four 
Inverted Pyramid of Language Proficiency, and 
Functional Tri-Section for Speaking 




Level 


FOBCtiOU 


Ceateit 


Accnracy 


Spealdog 
Levd 


Talk aooonylghed, attibules txpteaad, voce 
cocvcycd 


Topicsy subject ateas, activities 
aod jobs addressed. 


Acceptability, quality, aad accuracy of 
messace conveyed. 


5 


Fuoctioiis eqsivikat an EdDCiled Ntfive 
Speaker (ENS). 


All subjects. 


FerfbnMBce eqoivakat 10 Educated 
Native Speaker. 


4 


Able to tailoc liii(aate to & aadicace, oounseU 
penuade, Mfodaie, represcst a pojm o4f view, 
afid intcipcet for difiutmes. 


AB topics nocmally pertinet* to 
profeasiooal needL 


Nearly equiviJeat 10 ENS. Speech is 
exteasive, piacise, appcopciatt 10 every 
oocasioQ widi caly oocattoaal erron. 


3 


CaD oooveae io fixmal and icfocmal 
itoatioai, rcadve problem situadcias, deal 
with unfamiliar topics provide crplanirioos, 
deaoftw io deddl, suppoct cpi&iocft, asd 
hypotbefizc. 


Practical, social, pcofrstiooal, ard 
abctnct topics, paiticuLar iaterests, 
and yecial fields of competence 


Eziocs never ioteifere widi 
uodetsiandiflf aad rvely distub the 
ENS. Oaly sporadic errots ia basic 
structures. 


2 


Ahte to fully partkipaleiacMual 
ooftvautioni; cafi eiprcis bets; gjve 
iostrvctkxit; deaaibe, report 00, acwl provide 
aameioa about omat, patf, Md ftitute 
actfvitieL 


Coocrete topics sucb as own 
backcrouad, hmSiy, aad i&teresu, 
work tnvel,aodcttnee(eveats. 


Uodentaadable to NS nor used to 
dealioc widt ftxeifDers; aoeoetimes 


1 


Can create wid) the laofuagc; atk and lofwer 
questiooss participeie ia short oocvcnatioas. 


Everyday luivrval topics and 
courtesy requsemeots. 


lotetUfibk 10 an NS used to dealiflg 
with foretfoeis. 


0 


No fufictiotui ability. 


None. 


UatateQitft^le. 
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Pre-program levels of language are also reflected in predictor 
variables based on standardized qualifying examinations in Russian 
grammar and reading comprehension (Q-Grammar, Q-Reading), 
required of all program applicants. A further series of variables reflect 
results of the Modem Language Aptitude Test (MLAT). Separate 
scores on analytic, synthetic, and memoiy-based learning strategies 
(MLAT in, rv, V), are recorded along with an aggregate raw score, 
which can be compared as percentile to one of several norm groups 
consisting of US Wgh school graduates, enlisted servicemen, or US 
foreign service officers, MLAT test results are based on the success 
with which candidates manipulate samples of an artificial language 
("pseudo-Kurdish") and of English; the long-form MLAT (not 
included in the present data) also includes listening and alphabet 
mastery strategies. 

In addition to the batteries of pre- and post-program testing 
noted above, other post-program performance variables include grade 
reports on spoken and written Russian provided by each of the Soviet 
institutions (on a scale of 1 "failure"' to 5 "excellent") and a series of 
individualized attitudinal assessments, as rated by ACTR resident 
directors using a scale of 1 (low) to 5 (high), in the categories of 
"intellectual motivation," "natural ability to learn," "willingness to uce 
Russian," "cultural adaptability," "willingness to take advantage of in- 
country cultural opportunities," **ability to work in a group," and 
"leadership potential." It is understood by the investigators and 
program a(ininistrators that the attitudinal assessments are subjective, 
but nonetheless, such evaluations, like course grading, are a routine 
part of the assessment process in American higher education and are 
typical of the kinds of evaluations required annually of university 
faculty in recommending candidates for graduate and professional 
schools. 



2.2 The Pre-ZPost-Listening and Reading Tests 

Four 50-minutes .osts (tw^ forms each for reading and 
listening) are designed to be reliable as measures of reading and 
listening comprehension in the intermediate/high to superior range (1+ 
to 3 levels). Stimulus material in both sets of tests is drawn from a 
variety of nc^tural iang'^ ge use (authentic) sources, relying in 
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particular on materials taken from the mass media, Reading passages 
are printed texts in Russian, ranging from short passages to assess 
extraction of factual information, to larger texts designed to measure 
comprehension and analysis* Listening passages, administered by 
means of a tape recording, contain material in Russian spoken by both 
males and females at normal speed, such as news broadcasts, 
interviews, and conversations. Tne testing format is multiple-choice 
with responses recorded on a separate machine-^radeable answer 
sheet. All questions are printed in the test booklets and are based on 
information presented in the stimulus material itself. Specific subject 
matter knowledge is not tested; for example, there are no questions 
that bear on literary criticism, linguistic terminology, or, say, statistical 
analysis. Task assigmnents are stated in English prior to each Russian 
passage, so that the examinee knows in each case his/her purpose in 
listening/reading the text. The overall format is typical of standardized 
testing in the US and very familiar to American students. 

Whereas the boundary points between proficiency levels in 
both forms of the ETS Russian readings tests are the same. Form 2 of 
the listening comprehension is more difficult than Form 1, resulting in 
an adjustment in the weighing of raw scores:^ 



ETS Listening 


Raw Score 


Raw Score 


Proficiency Level 


Form I 


Form 2 


3 


44-57 


43-57 


2+ 


41-43 


39-42 


2 


37-40 


33-38 


1+ 


31-36 


27-32 



2.3 The Pre- and Post-Program 
Oral Proficiency Intemewsio 

The OPI is a testing procedure designed to measure a wide 
range of functional speaking abilities from novice ("0") to that of an 
educated native speaker ("5"). In the course of the interview, the 
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Student converses with one or two trained testers on a variety of topics 
for 10 to 40 minutes, depending on the level of the student's 
proficiency. The resulting speech sample, which is normally recorded 
on audio or video tape for subsequent verification or analysis, is then 
rated on a scale ranging from 0 (no practical functional ability in the 
language) to 5 (ability equivalent to that of a well-educated native 
speaker). "Plus" ratings (0+, 1+, 2+, 3+ etc.) are given to students who 
substantially surpass the requirements for a given level but fail to 
sustain performance at the next higher level. The rating is a single 
global indicator of the learner's probable ability to communicate in the 
target language/culture from the point of view of language functions, 
context/content areas, and acceptable levels of granmiatical accuracy 
(Refer to Exhibit Four, the Functional Tri-Section for Speaking). As 
contrasted with an achievement test, which is based on specific 
material covered in a particular course of study, the proficiency test 
compares the student's ability to that of an educated native speaker 
using the language for actual communicative purposes. 

The OPI was originally developed by the Foreign Service 
Institute (U.S. Department of State) as a means of verifying the 
foreign language skills of diplomats abroad. It was later adopted by 
the American Council on the Teaching of Foreign Languages and the 
Educational Testing Service for use in the American educational 
system, where it has received broad acceptance in recent years. Its 
relatively high degree of reliability is due in part to the rigorous 
training seminars which all OPI testers must undergo in order to 
become certified and to maintain themselves as testers. The OPI offers 
a degree of face validity to as well, in that it requires the candidate to 
speak in a realistic conversational setting. The content validity of the 
OPI is maintained by reasonably rigorous testing protocols which 
require trained interviewers always to test repeatedly for the functions, 
content, and accuracy that characterize each level. 

Unlike the listening and reading tests, the OPI requires that at 
least one tester spend typically from 20-30 minutes to elicit a rateable 
sample from each candidate. The OPI is, therefore, a more time- 
consuming procedure than many traditional testing instruments; it is 
reminiscent of oral testing long established in Russian and Soviet 
academic examinations. The incidence of unrateable samples from 
this procedure is relatively low; f irthermore, certified testers 
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independently assign to the same live or recorded OPI the ''^e rating, 
or, occasionally differ by only a "plus" point. 

The five-point proficiency scale is best visualized as a three- 
dimensional model resembling an inverted pyramid, with the novice- 
level of proficiency at the bottom and the native ("5") level at the top. 
(See Exhibit Four, Inverted Pyramid of Language Proficiency) The 
typical acquisition time of an English-speaking learner of Russian of 
above average/average language learning aptitude (for example, as 
measured by MLAT) is included in Exhibit Four. 



3*0 Data Analysis 

In this section we present some preliminary results on 
variables related to gains in language competence for students in the 
four-month ACTR programs. From the policy point of view, our aim 
is to establish the characteristics of students who gain and to isolate 
variables that discriminate between students who gain and students 
who do not. As a statistical study of correlates of gain, our approach is 
analogous to that of epidemiological studies of diseases and 
pharmacological studies of the effectiveness of drugs. In this mode, 
men over a certain age might be advised by their physician to take an 
aspirin every other day on the grounds that it lowers their risk of heart 
disease, even if the mechanisms are not known and even if some 
individuals may not derive any benefit. Analogously, in the present 
study our perspective is that of a program director and our emphasis is 
on effective placement and program management on a statistical basis. 
The issues involved concern placing students in programs which best 
serve their individual needs, such as at what stage of language 
learning can students take best advantage of study abroad? Which 
students should go, and when? What gross characteristics of programs 
and experiences are the most productive for given learners? Thus, on 
the basis of results such as ours, students with good grounding in 
grammar but little speaking proficiency might be advised to study 
abroad at a particular time and placed in particular programs. 

From the point of view of the acquisition process itself, our 
analysis provides the factual context for more detailed scientific 
studies of what students learn and how they learn it. Our aim here is to 




On Evaluating Language Proficiency Gain in Study Abroad Environments 
Richard D. Brecht/Dan E. Davidson/Ralph B. Ginsberg 

establish robust relationships that require explanation and to suggest 
hypotheses that might guide subsequent research. Broadly speaking, 
explanations of relationships fall into three categories, all of which are 
represented below: specification of other variables ^vhich could 
account for the connection, e.g. differences in language gains between 
men and women might be accounted for in terms of tlieir ed?icational 
goals or their overall academic performance; specificatioa of causal 
processes at a microlevel that produce the observed relationships, e.g. 
differences in gains between men and women might be accounted for 
in temas of differentials in the structure of conversations with native 
speakers which affect the learning process (this would be analogous to 
microbiological studies of how viruses affect cells); and specification 
of artifacts in measures and statistical procedures used in the analysis, 
e.g., variables may lack explanatory power because the oral 
proficiency interview is not sensitive enough, at a particular levels of 
proficiency, to capture the benefits of four months study abroad, or 
because a gain of a level may be much easier at the intermediate than 
at the advanced level. Although ail of these types of explanation in the 
final analysis call for more and better data, we shall see that the results 
based on the data available are pertinent to many of the questions 
commonly raised in the field and strong enough to draw conclusions 
that are not purely speculative. 



3.1 Analytical Methods 

The results we shall present were for the most part derived 
using regression analysis. Regression analysis is a family of statistical 
techniques which permits assessment of the effects of several factors 
simultaneously, and, more importantly, produces estimated 
relationships between the criterion (dependent variable) and its 
determinants (independent variables) which are purged of effects of 
other variables which could affect them. This is not the place to go 
into the technical details of our statistical methods. Indeed we shall 
only present a qualitative summary of our results here, leaving fuUer 
discussion for a forthcoming study. Intuitively, the motiv^ition of our 
methods and an indication of how the results can be interpreted are 
captured in the old cliche that correlation does not imply causality. 
The more firemen there are at a fire the worse the damage, and the 
more storks there are in an area the higher the birth rate, but that does 
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not mean that firemen cause damage or that storks bring babies. 
Clearly in the former the severity of the fire, and in the latter whether 
an area is rural or urban, must be taken into account, i.e. brought 
under control, before any meaningful relationships can be stated. 
Similarly, to take a somewhat whimsical example from Mosteller and 
Tukey, in examining the relationship between performance on a 
French dictation test and body weight, one would find a strong 
positive relationship if the sample contained children ranging in age 
from eight to sixteen (with the older children being heavy and more 
accomplished in French and the younger children being light and less 
accomplished), a strong negative relationship if the sample were 
restricted to sixteen-year-olds but contained a mixture of French 
(lighter) and American (heavier) students, and a weak negative 
relationship in a sample of sixteen-year-old Americans containing 
girls (on average lighter and better at French) and boys. Again, in 
order to interpret the observed relationship, the population in which it 
occurs must be specified and key variables related both to French 
proficiency and body weight (age, nationality, gender, as the case may 
be) isolated and controlled. Regression analysis is a way of exercising 
this needed control statistically, purging estimated relationships of the 
effects of many extraneous factors simultaneously, and leading to 
estimated effects which, while not guaranteed to be free of spurious 
effects, at the least cannot be explained on the basis of any other 
variables that are explicitly included in the analysis. 

In our investigation of the determinates of gain during study 
abroad we examined all three language proficiency measures for 
which pre- and post-tests were available, viz. oral proficiency as 
measured by the OPI, and listening proficiency and reading 
proficiency, as measured by the ETS tests. Gain, the dependent 
variable in our analysis, is defined as the difference between before 
and after ratings. In the case of the OPI, where ratings are in terms of 
discrete categories, several binary criteria based on changes were used 
(see below). In the case of the ETS tests our analysis is based on the 
raw scores. Because the before and after tests are not equivalent in 
difficulty, ttie dih ^rence in scores is not meaningful per se, but the 
estimated effects in the regression analysis are still valid. As 
independent variables, i.e, factors that could account for differentials 
in gain, we considered all of the variables in the ACTR database 
described in the previous section. Regression equations (logit 
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regressions in the case of the OPI, ordinary least squares in the case of 
ETS listening and reading) were estimated to determine effects, 
controlled for the influence of other variables, based on all cases with 
valid data. (The total number of observations varied from analysis to 
analysis, since ETS tests were administered starting in 1986 while 
OPIs were administered to everyone starting in 1984, and since in 
both cases there were a small number of students for whom scores 
were not available.) 



3.2 Oral Proficiency 

We look first at factors related to gains in oral proficiency as 
measured on the oral proficiency inter/iews (OPFs) taken at the pre- 
program departure orientation and in the Soviet Union just at the end 
of the four-month program. Exhibit Five shows the overwhelming 
effect of initial proficiency level on the likelihood of gain, an effect 
which is equally strong no matter what other variables are controlled. 
For those starting at 0+ (novice high) everyone improves; among 
those who begin at 1 (intermediate), by far the largest group, only 
14% do not gain and almost 30% gain two or more points, reaching 
the advanced category; among the l+'s (intermediate highs), 28% 
show no gain although again more than 30% gain two or more; 
among advanced students (2, 2+, and 3) abou' 55% show no gain, 
although many still make significant improvements. These results can 
be explained partly by the testing artifact noted above that as one 
moves up the OPI rating scale, ever expanding language skills are 
required (so that the scale points (0+, 1, 1+, 2, etc.) are further apart as 
one moves up), and partly by the general nature of learning curves for 
complex skiUs, where a given increment requires increasing time and 
effort as proficiency increases. Whatever the reason, it is essential to 
control for initial level in examining the effects of any variable that 
nfiight be related to gain. 

With Exhibit Five in mind, three analyses of factors related to 
OPI gains were run. In the first we used the straightforward criterion 
of no gain vs. gain (i.e. those in the first column of the table vs. the 
rest), with initial level controlled quantitatively in the logit 
regressions. In a second analysis we tried to compensate for the fact 
that it is easier to gain at the lower levels than at the higher by 
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equating students who started at 0+ and 1 and gained only a half a 
level (1 point) with nongainers at the higher levels, thus contrasting 
students gaining 2 or more starting at 0+ and 1, and students gaining 1 
or more at 1+ and above, with the rest (right vs. left of the heavy line 
in the table). In a third analysis v/e confined our attention to students 
who were initially at 1 (the group large enough to study by itself), thus 
automatically eliminating any effects of initial level, and examined 
factors related to gains of two or more (i.e. gains from intermediate to 
advanced on the OPI scale). 

Our analyses indicate that only a few variables in the database 
are related to OPI gain when proper control is exercised, no matter 
what criterion is used. (The logit regression results are confirmed by 
tree structured regression, a ver}' different statistical technique, to be 
reported in a subsequent publication.) Two variables are highly 
significant on all criteria: 

• the higher the scores on the ACTR qualify- 
ing reading test, the more likely a student is 
to gain, and 

• men tend to gain more than women. 

The effect of good reading skills on oral proficiency gains is 
particularly strong among initial Vs moving into the 2 category (OPI 
analysis 3 above). As expected, initial OPI level had an extremely 
strong effect on gain, even with all the other variables controlled, 
which implies that the nature of the scale or the learning curve, rather 
than differences in preparation or motivation, are at work. With the 
stricter, second criterion, requiring gains of 2 or more at the low 
levels, the effect of initial OPI on average disappears, implying that 
the criterion adjusts for initial level. 

On the simple gain vs. no gain criterion, two further variables 
are significant: 

• students in earlier years of the program 
(1984) gain more often than students in more 
recent years (1989), and 

• students with knowledge of other (non- 
Slavic) languages gain more than students 
whose only foreign language is Russian. 
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Exhibit Five 
Gain in OPI by Initial Level* 



Gain 



(Based on Post-Program OPI Rating)t 



Program 
or I Rating 



0 or loss ^ 


2 


3 or more 


Total 


Oi- 




^ 

Ht 

^iL— — 


10 

sm 


2 

iS 


26 


2 


35 


151 


56 

[(22) 


" 1 

J2 


263 


2+ 


27 
(28) 


39 
(41) 


29 
(30) 


1 

(1) 


96 


2 


23 


19 






42 




J55) 
12 


8 




,1 




20 




97 


231 


95 


21 


444 



•Heavy line defines the secoad criterion of gain. 
tNumber (percent) of students (subjects in sampling) 
$Given a scale of: 



0+ 
1 
1+ 
2 

2+ 
3 



• f ..1 » in the oresent analysis indicates movement from one 
a gam of 1 m tbe presenv ^o^j , of 2 



21 



120 



On Evaluating Laaguage Proficiency Gain in Study Abroad Environments 
Richard D. Brccht/Dan E. Davidson/Ralph B. Ginsberg 



There are also significant differences in simple gains between 
the five institutes in which students take courses, although they are 
difficult to interpret on the basis of difference in the formal, in-class 
programs. Among the students starting at 1 there is some evidence tfiat 
younger students are more likely to make substantial gains than older 
students, although the effect is not highly significant statistically. 
Rounding out the main analysis, there is some indication that the more 
lab hours and the less classroom hours students have had in their 
previous courses the better they do. These results are, however, 
somewhat ambiguous because these variables are highly correlated 
with one another and vvith other key variables in the analysis. Since 
both class and lab hoxurs are important determinants of initial language 
levels, interpretation of their effects must be based on factors 
associated vvdth education but not measured by any of the tests. Finally 
with regard to oral proficiency, tfiere is intriguing evidence that the 
resident directors' ratings of motivation and attitudes are significant 
indicators of gain. In particular, on all criteria, leadership potential is 
clearly positively related and ability to work in groups clearly 
negatively related to gain; native ability, willingness to use Russian, 
and cultural adaptivity have positive but less certain effects. These 
results must be considered preliminary, however, since resident 
director ratings are missing in the database in some cases and since 
these variables are very highly intercorrelated. 

In interpreting the regression results it is important to 
remember that the effects attributed to variables noted above hold over 
and above those of any other variables in tiie analysis. Or, to put it 
slightly differently, the effects reported here cannot be explained by 
any other variables in the database. Thus, for example, better readers 
by and large have higher oral proficiency scores, but the results say 
that at every level of initial proficiency a better reader is more likely to 
gain than a poorer one. Similarly, there are many differences between 
men and women in academic achievement, motivation, etc., but the 
somewhat counter-intuitive result that men do better than women in 
the programs we study holds after all other differences attributable to 
variables we have measured (language aptitude, major field of study, 
previous preparation, mastery of other languages, etc.) have been 
ruled out as possible explanations. The explanation of gender 
differences must, accordingly, lie in factors we have not measured, 
such as differences in the nature of language learning experiences that 
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men and women have while in the Soviet Union. We shall return to 
this point below* 

It should also be noted that many variables which one might 
think a priori are important turn out not to be associated with OPI 
gain in our data, once the key factors mentioned above are taken into 
account and controlled. These include: language aptitude, as measured 
by the MLAT*s, major field of study, highest degree earned, and 
participation in a previous immersion program. 



33 Listening Proficiency 

We turn now to gains in listening proficiency as measured by 
changes on the ETS test. For the most part, students on the ACTR 
programs start at the intermediate level (1 and 1+), although some are 
in higher categories. As with the OPI, there is an extremely strong, 
negative relationship between initial level and gain (with a t-statistic 
in the regressions over 10). Controlling for initial level, several 
variables are significantly related to gain, most of which were found to 
be significant for oral proficiency as well: 

• younger students gain substantially more on 
average than older students; 

• men gain more on average than women; 

• smdents with knowledge of othcr» non-Slavic 
languages gain more on average than 
students with no other languages; 

• students gain more on average if they have 
been involved previously in an in-country 
immersion program; 

• students with high language aptitude, as 
measured by the MLAT, gain substantially 
more on average than students with low 
aptitude; 

• students with high qualifying reading scores 
gain substantially more on average than 
students with low scores. 

There are also quite significant differences associated with the 
institute in which the student is enrolled, but as with the OPI these 
effects are difficult to interpret without further analysis. TTiere is some 
indication that lab hours arc positively related to gains, but the effect 
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is not statistically significant by the usual criteria and should therefore 
not be interpreted here. Save for the difference between men and 
women, which is consistent with the results on the OPI, all of the 
relationships noted above are in the expected direction. 

3.4 Reading Proficiency 

Finally, we look at changes in reading proficiency. Over 40% 
of ACTR students are already advanced readers when they begin the 
program, but there is still room for gain on the ETS test. In addition to 
initial level, which in this case too has an extremely strong effect, 

• language aptitude (MLAT) is very strongly, 
positively related to gain; 

• scores on the ACTR qualifying grammar test 
are also positively related to gain, although 
this effect is difficult to separate from a 
similar positive effect of the qualifying 
reading test; 

• knowledge of other (non-Slavic) languages is 
positively related to reading gain. 

There are also strong differences among institutes. Even with reading, 
the sex effect favors men, although its magnitude is not quite large 
enough to reich statistical significance. 

In concluding this presentation of results, let us reiterate that 
the effects noted here are genuine in the sense that they cannot be 
accounted for by any other variables in the study, and they must stand 
to the extent that no obvious variables characterizing students before 
entry into the ACTR program (at least), are left unmeasured. 
Moreover, variables in the ACTR database which do not have 
statistically significant effects are in all likelihood unimportant 
predictors when the significant variables are taken into account. Of 
course, there are many variables affecting gain which we have not 
measured, in particular variables characterizing what happens during 
the student*s pexiod of study in the Soviet Union. Measuring these 
would help to give us a much firmer grasp of the nature of the learning 
process and of the mechanisms by which the variables in this study 
have their effects. We return to this discussion in the concluding 
section. 
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4.0 Discussion 

Significant gains in language proficiency are achieved by a 
wide variety of American college-level learners of Russian as a result 
of their in-country immersion experience. Of particular interest in this 
connection are gains resulting in post-program oral and listening 
proficiency in the 2-range ("advanced") or higher, skill levels rarely 
attained by graduates of stateside B.A, programs in Russian, On the 
OPI (see Exhibit Five) 28% of the initial Is make the large jump t ) 2 
^d above and of the 1+s fully 72% cross the 2 barrier. On the ETS 
listening test (with scores converted to proficiency ratings according 
to the table in section 2.2), 45 % of the 280 students rated as 1 at the 
start of the program (the bulk of the sample) make the big leap to 2 
and above, and 77% of the 48 initial 1+s cross the 2 barrier. Very 
substantial gains in reading are common as well. Twenty-four percent 
of initial Is and 60% of initial 1+s reach 2 and above. Of particular 
interest are the sigrdficant number of students who reach the 3 -level 
("superior") in reading - 8% of initial Is, 17% of initial 1+s, 45% of 
initial 2s, and 57% of initial 2+s - so that at the end of the prograni 
fully 27% of the 316 students not there to begin with reach the hi^est 
possible level of this test, a level considered adequate for regular 
professional activity requiring the use of Russian . All in all, 
comparable levels in all tiiiree skills are obtainable only by further 
immersion study and/or comparable programs. (We shall analyze the 
effects of Soviet Academic Year Programs in a forthcoming paper.) 

4 J Oral Proficiency 

Among key determinants of gain in Russian, qualifying 
grammar and reading achievement scores show significant predictive 
value for oral proficiency, reading proficiency, ana listening 
proficiency alike. Higher levels of control of basic grammar and 
reading skills, as measured by Q-Grammar/Q-Read, correlate 
positively with gain in all three skills. Although the data indicate that 
the lower the initial proficiency level the greater the gain — an effect 
which is partly an artifact of the proficiency scale (see Exhibit Four), 
and partly a result of the learning curve itself — the significant fact is 
that at any given initial level the better prepared one is in 
reading/granmiar the more one tends to gain. This finding and the size 
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of the sample on which it is based has significance for the on-going 
debates in the foreign language acquisition field (especially following 
Ifiggs and Clifford, 1983) concerning the role of explicit granunatical 
knowledge in the development of communicative stalls, as well as for 
our imderstanding of the interrelationship of the several skills in the 
acquisitional process. Given its importance, additional research should 
be devoted to replicating the results of the present study. 

With respect to the usefulness of the numerous testing 
instruments involved in the present study, it can be concluded that the 
now 30-year-old Modem Foreign Language Aptitude Test (MLAT) is 
a valid predictor of success in developing reading and listening sldlls 
in Russian; the MLAT, however, does not serve to predict oral 
proficiency gain as measured by the standard OPL Further analysis of 
the role of attitudinal factors affecting gain (especially leadership 
ability, but also cultural adaptability, and motivation) is important, 
particularly the extent to which these factors can be seen to 
compensate for deficiencies in other areas also known to affect 
proficiency gain. 

The dramatic increase in opportunities for study in the USSR 
during the past five years has made it possible to attract a more diverse 
student population to the programs with initial skill levels that more 
closely approximate those of the typical US undergraduate major. In 
the early years of the present study, graduate students and teaching 
assistants made up a large percentage of all ACTR in-country 
semester placements, while Soviet admission regulations excluded 
from participation in these programs virtually anyone other than 
current and future teachers of Russian. Although there does not seem 
to be any relationship between college major and gain in our data, 
further analysis of the performance of program participants by 
disciplines as well as by host institutions is needed before it will be 
possible to interpret the program-year effect. It would be difficult to 
deny that the opening of Soviet society under Gorbachev has made 
study in that country more stimulating intellectually for American 
students than ever before, even as it has multiplied the possibilities for 
significant cultural immersion within Soviet society. In short, all our 
evidence indicates that the programs in recent years are academically 
stronger than in years past; the explanation for the "year-effect" 
remains elusive. 
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4.2 Reading and Listening Proficiency 

The above results indicate that gain, as measured by OPI, ETS 
Listening, and ETS Reading, respectively, is positively correlated with 
a students* knowledge of another (non-Slavic) foreign language. 
(Most often, French, German or Spanish.) Controlling for language 
aptitude (MLAT) and for motivation (as indicated by the resident 
director evaluations), one plausible interpretation is what might be 
termed the "skilled second language learner" effect, which involves 
more efficient use of specific kinds of cognitive strategies for 
comimunicating in natural environments and for expanding one's 
lingmstic knowledge on the basis of this communication.^^ Clearly, 
more data concerning the influence of previous second language 
experience on learning in and outside of class is needed, data which 
would include ethnographic observations on behavioral differences 
between skilled and non-skilled language learners particularly in real 
conununication tasks in natural environments. 

Another factor in the acquisitional process, which our data and 
analysis reveal, is the effect of a previous immersion experience on 
the rate of gain in listening skills. The beneficial effect of a previous 
immersion experience seems logical, even if little investigated to date. 
Students who have had significant experience in communicating 
successfully under uncontrolled conditions (i.e., outside of the 
classroom with native speakers) might well be expected to hav:^ 
developed specific comprehension strategies transferable to the in- 
country environment of the ACTR programs. As is the case with 
students with previous second language experience, more information 
will be required to determine how and to what extent learning 
opportunities are expanded for students who retum to an immersion 
environment. 



43 Other Observations 

A further notable finding of this study is that, when other 
important factors are controlled, the gains as measured by the OPI and 
the ETS Listening tests are significantly higher for males than for 
females. A comprehensive analysis of this phenomenon requires 
ethnographic study of conmiunicational interactions in-country, both 
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in and outside of classroom, to determine whether culturally-defined 
gender roles have an impact upon communication and upon learning 
opportunities. 

In the area of reading proficiency gain, younger students 
experience somewhat greater success than their older colleagues. Age 
traditionally is assumed to play a role in second language acquisition, 
but mostly with regard to children versus young adults.^^ Our study 
does not include the acquisition of a second language by children; it 
deals with learners in the 19-39 year age group and stands, therefore, 
outside of the bulk of SLA literature on age differences in the 
acquisitional process, awaiting further investigation and/or repUcation. 

5.0 Next Steps 

The results of the preceding analysis correlating gain with a 
range of independent variables reveal significant information 
concerning SLA in the study-abroad environment. However, as the 
database grows and as more variables and programs are added, it is 
clear that much more analysis and interpretation needs to be done. In 
addition, as the pre-'^nt project begins to yield important insights into 
the impact of study-abroad, the questions provoked by these results 
have led to a second major project in the overall study of SLA in 
study-abroad environment, undertaken by the NFLC with the support 
of the Ford Foundation and in cooperation with ACTR. The new 
project hopes to provide more discrimination both in the evaluation of 
"gain" and in the input and processing factors which characterize 
study-abroad. 

Given its decreasing ability to reflect changes at the advanced 
level and beyond, there is little hope that the OPI can serve as a 
criterion for systematic research. There is a need for more 
discriminating measures of skills in the advanced range before 
quantitative studies can be fully exploited. To determine factors 
associated with gain during study abroad, the structured and 
unstructured learning environments must be carefully observed, both 
in terms of actual L2 input and in temis of individual and socially 
distributed learning processes. The Ford Foundation has funded the 
next project in the ACTR/NFLC long range analysis of study abroad. 
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Ethnographic data will be integrated with and conditioned on 
variables found to be si^uficant in the present study, including gender, 
previous immersion, taiowledge of other languages, and control of 
basic grammar. 



^ The ACTR.data base, begun in 1976, is growing at the present rate of 
approximately 120 student profiles a semester. Plans are to extend the base to ten- 
month and summer students as well. 

2 As a rule, administrators of most 'oreign language programs in American 
schools and imiversitics are themselves untrained in second language acquisition and 
often unaware of SL research. For the most part, the chairs are literature specialists 
or linguists, who may or may not heve direct experience in the language teaching 
classroom. In the United States, the deans and other upper level administrators to 
whom they report and from whom tfiey receive budgets are often scholars who may 
lack personal experience with foreign languages. 

^ An examination of the factors internal to study-abroad SLA can be 
revealing and suggest the need for a contrastive study with immersion domestic 
programs. 

^ Swaffar (1989) characterizes these changes as a shift in paradigm, in the 
Kuhnian sense. See Kuhn, Thomas. The Structure of Scientific Revolutions. 2nd ed. 
Chicago: University of Chicago Press, 1970. 

^ .\s Ellis (1986:18) puts it, **A theory of SLA is an attempt to show how 
input, internal processing, and linguistic output are related.*' See Ellis, Rod. 
Understanding Second Language Acquisition, 1986, Oxford University Press. 
Exhibit One is an elaboration of similar schemas in Ellis 1986. 

^ The present study shares elements of approach with a recent study 
conducted by the European Science Foundation; both studies are empirical, multi- 
year, cross-sectional and longitudinal investigations of the acquisition of language in 
an immersion environment. However, while the ESF project investigates 
spontaneous second language acquisition on the part of migrant workers in Western 
Europe, we are focused on advanced-level SLA in structured learning programs with 
an in-country immersion component. In contrast to the ESF project, which measures 
language gain by means of a series of non-standardized instruments and 
observations, the ACTR/NFLC study takes as its starting point the set of 
measurement instruments employed across the academic and governmental language 
teaching profession. The advantages of this more traditional approach are obvious. 
Equally important, though, is the fact that this approach provides an unprecedented 
opportunity to evaluate the testing instruments themselves, or to determine just what 
they are testing. Sec also Clive Perdue, ed. Second Language Acquisition by Adult 
Immigrants: A Field Manual Rowley, MA: Newbury House Publishers, 1 984. 

^ See Dan Davidson, "Assessing Language Proficiency Levels of 
American Participants in Russian Language Programs in the Soviet Union," Russian 
Language Journal, XXXVI, No. 125, 1982, 221-232. 
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^ Of the 3000-odd two and four-year colleges in the U.S,, approximately 
475 offer regular Russian language training, of which, approximately 245 include 
undergraduate major programs, 75 M,A.-levcl programs, and 35 Ph.D.-level 
programs in language and literature. Summer "intensive programs" are reasonably 
common at B.A. and graduate-level institutions and provide comparable contact 
hours during two summer months as would normally result from a two-semester 
through-the-year course (i.e., 100-140 hours). A special role in accelerating the 
lariguage training of American students has been played by the intensive-immersion 
tj pe, providing in the vicmity of 250-contact hours of training within the controlled 
environment of an "around-the-clock" cultural enhancement and residential 
program. The Middlebury Suncuner School is the oldest such program in the US, 
with similar long-standing inamersion programs in place at Bryn Mawr College, 
Indiana University, and Norwich University. These sunmier immersion programs 
admit qualified students from throughout the US and abroad. 

^ Detailed information on the methods of calibration and norming of the 
ETS advanced-level reading and listening tests used in this study are reported in the 
ETS publication, Russian Proficiency Test. Test and Score Manual, Princeton, New 
Jersey, 1986. Of the approximately 500 American university students taking part in 
the norming of the ETS tests in 1985-6, only 2% tested at the "superior"' level (3) in 
listening, while 12% tested at that level in reading; of these, more than half were in 
their fifth year of formal study of Russian or beyond. While 78% of the norm group 
scored below 1+ on the listening exam, including nearly half of Aose in their fifth 
year of study or beyond, only 31% scored below 1+ on the reading exam, including 
12 of the 42 persons in their fifth year of study. In-countty performance data in 
reading, and especially in listening comprehension development, is of particular 
interest given the obvious inadequacies of stateside training in these skills. 

The Oral Proficiency Interview (OPI), as adapted for use in US 
academic settings, has been well described in the literature, especially after 1982: 
Educational Testing Service. ETS Oral Proficiency Testing Manual Princeton, 
1982; (Pardee Lowe, Jr., *The ILR Oral Inter\dew: Origins, Applications, Pitfalls, 
and Implications," Die Unterrichtspraxis 16 (1983), 230-232; Judith Liskin- 
Gasparro, "The ACTFL Proficiency Guidelines: An Historical Perspective," 
Teaching for Proficiency, the Organizing Principle, ed. Theodore V. Higgs, 
Lincolnwood, IL, 1984, 11-42; also, see Claire Kramsch, "Proficiency versus 
Achievement: Reflections on the Proficiency Movement," ADFL Bulletin, 18, No. 1, 
September 1986, 22 - 24. Lyle Bachman and Sandra Savign on, *T he Evaluation of 
Communicative Language Proficiency: A Critique of the ACITL Oral Interview," 
Modem Language Journal 70, 1986, 380-390; Heidi Byrnes, "Speech as Process," 
Foreign Language Annals, 20, No. 4 , 1987, 301-310. Russian-specific proficiency 
guidelines were published by ACTFL under USED contract in 1984 (preliminary) 
and released along with generic guidelines in 1986: American Council on the 
Teaching of Foreign Languages. ACTFL Proficiency Guidelines, Hastings-on- 
Hudson, 1986. See also Dandonoli, P., G. Henning, "An Investigation of the 
Construct Validity of the ACTFL Proficiency Guidelines and Oral Proficiency 
InX&Tvitv/,'' Foreign Language Annuals, 23,}^o, 1, 1990, 11-22. 

R. Brecht, D, Davidson, R. Ginsberg, "Advanced Acquisition of 
Russian in a Study Abroad Environment. A Longitudinal Study of Skills 
Development," forthcoming. 
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Published work on the subject of communication strategies refers to 
"reduction'* and ''achievement" strategies, the former referring to avoidance conduct 
when faced with a commimication "breakdown*' and the latter to compensation. The 
assumption that learning is enhanced by compensation strategies which attempt to 
keep to the task at hand and to elicit the required knowledge. See Ellis 1986 (187 ff,) 
for a discussion of the literature on the subject. 

13 See the references in Ellis 1986: 104 ff. 
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